User engagement on
NGO Facebook pages

Effects of post sentiment measures over time

Nils Gustafsson & Nils Holmberg

Research questions

RQ1: How do Swedish nonprofits’ use of emotional expressions on Facebook change over time?
RQ2: What are the effects of emotional expressions on user engagement in Facebook posts from Swedish nonprofits?

ca 70 000 posts, after cleaning
from 125 pages (e.g. WHO)
time period of 2014-2019
downloaded using CrowdTangle

Engagement index: sum of likes, comments and shares

Sentiment dictionary

content coded for sentiment using the AFINN dictionary
better results with machine learning ?
quantitative content analysis vs computational

some caption

Sentiment measures

ca 70 000 posts from 125 organizational pages
time period of 2014-2019
text pre-processing, emojis converted to text descriptions
sentiment analysis on all text content

date of posting
number of followers
post language

some caption

Sentiment analysis

content coded for sentiment using the AFINN dictionary
words and emojis assigned valence value of -5 to +5
valence indicate positive or negative sentiment (0 being neutral)
valence squared indicates intensity of sentiment

some caption

Sentiment over time

sentiment normalized within organizations
normalized sentiment aggregated per month
positive relationship between sentiment and time?

some caption

Engagement and sentiment

engagement index: sum of likes, comments, shares
engagement index normalized within organizations
normalized engagment aggregated per month (mean)
positive relationship between engagment and sentiment?

some caption

Effects on engagement

positive effects
- post sentiment valence
- post sentiment intensity
- number of followers at time of posting

negative effects
- time, month number 1-60

type of organization?

Linear mixed effects model with organization as random factor

Variables by organization type

1=culture and recreation
2=education and research
3=health
4=social services
5=environment
6=development and housing
7=law, advocacy an politics
8=philantropic intermediaries and voluntarism promotion
9=international

Engagement, mean over time

big differences between organizations
green: type health, large org
red: type international, small org

Engagement, trend over time

relatera sina akademiska
tillämpa teorier inom
utveckla erfarenheter att

Engagement measures

engagement index, dependent variable

1. e_index_mean: per post e_index average per month (“raw engagement scores”)
1. e_index_mean_norm: normalized per organization values of e_index_mean (1. above)
1. e_index_norm_mean: per post e_index normalized per organization average per month
1. e_index_rel_mean: per post e_index relative to follower count average per month

Engagement trend by organization type

relatera sina akademiska
tillämpa teorier inom
utveckla erfarenheter att

Engagement measures by organization type

relatera sina akademiska
tillämpa teorier inom
utveckla erfarenheter att

Thanks!

Presentation

Nils Holmberg
Strategic Communication
Cognitive Science